๐Ÿš€ Pipeline Results Report

Generated: /root/verbalize-sampling

Experiments: 5

Metrics: diversity, ngram, creative_writing_v3, length

๐Ÿ“‹ Experiment Configurations

NameTaskMethodModelResponsesTemperature
direct (samples=1)poemdirectgoogle/gemini-2.5-flash300.7
multi_turn [strict] (samples=5)poemmulti_turngoogle/gemini-2.5-flash300.7
sequence [strict] (samples=5)poemsequencegoogle/gemini-2.5-flash300.7
structure_with_prob [strict] (samples=5)poemstructure_with_probgoogle/gemini-2.5-flash300.7
combined [strict] (samples=5)poemcombinedgoogle/gemini-2.5-flash300.7

๐Ÿ“ Generation Examples

direct (samples=1)

Method: directTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
Truth! I shall speak you if I am in habit of listening to you,a quiet echo in the soul's deep hue.For often, clamor drowns your subtle plea,and busy minds forget to set you free.I chase the fleeting, grasp the shallow gleam,distorted visions in a waking dream.My ears attuned to flattery's soft embrace,or urgent whispers of a hurried pace.The world, a siren, calls with gilded tongue,and honest echoes are too rarely sung.But if I pause, and still the inner tide,and let your gentle current be my guide,if I incline my spirit, soft and low,to hear the seeds of understanding grow,then from that stillness, rooted, clear, and bright,my voice will carry your unwavering light.For silence nurtures what the heart has heard,and true conviction shapes the spoken word.
Method: directTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
Truth! I shall speak you if I am in habit of listening to you,a quiet echo in the soul's deep hue.For often, you arrive in whispers soft and low,lost in the clamor, the hurried ebb and flow.My ears, attuned to the world's insistent hum,miss the gentle cadence, the wisdom yet to come.I chase the glint of surface, the immediate bright,while deeper currents beckon, veiled in fading light.So teach me, Truth, to still the restless mind,to seek the silent spaces, the peace I hope to find.To let the clamor cease, the urgent voices fade,and in that quiet stillness, your clear message made.Then, from that wellspring, pure and undefiled,my voice will rise, no longer running wild.No longer echoing falsehoods, or shadows of deceit,but speaking you, Truth, perfectly complete.For only then, when I have truly heard your call,can I stand firm and let your pure words fall.
Method: directTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
Truth! I shall speak you if I am in habit of listening to you,a whisper, a rustle, a soft, insistent plea.Not the booming decree, the thunderous, proud view,but the quiet unfolding, the slow, dawning sea.For often you call, in the hush of the night,in the tremor of doubt, the flicker of thought.And I, with my armour, my blinkered, bright light,have deafened myself to the lessons you've taught.Iโ€™ve chased the loud echoes, the comforting lies,the glint of approval, the popular sway.Ignoring the wisdom that gleamed in your eyes,the subtle corrections that guided my way.But now, I endeavor, to silence the din,to open my spirit, to hear your soft hum.To welcome the quiet, the voice from within,and let your pure cadence, finally become.Then, when I speak, it won't be a decree,but a resonance deep, a reflection so clear.For the words will be yours, flowing truly through me,the echoes of listening, dispelling all fear.

multi_turn [strict] (samples=5)

Method: multi_turnTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!A titanโ€™s breath, a dragonโ€™s chase,From earthly bonds, the swift release,A silver needle, leaving trace.Through velvet night, a searing gleam,A whispered wish, a vibrant dream,Of worlds unknown, a cosmic stream,Igniting hope, a radiant beam.The hum of power, a rising roar,Beyond the clouds, forevermore,Each fiery plume, a story spun,Of journeys dared, and victories won.A fleeting star, a transient sign,Of human will, and grand design,To pierce the veil, to touch the high,And paint the darkness with a sigh.Then fading slow, to pinprick bright,A memory etched in endless night,The echoes linger, soft and deep,Of promises the heavens keep.
Method: multi_turnTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!A crimson flower, suddenly blooms,Unfurling petals through the glooms,Dispelling shadows, chasing dooms.A whispered prayer, a silent plea,For distant shores, for liberty,A burning arrow, wild and free,Across the cosmic, boundless sea.The engine's thrum, a beating heart,Tearing the darkness, right apart,A fleeting moment, a work of art,A brand new journey, set to start.From earthly chains, a grand escape,A fiery dragon, takes its shape,Across the void, no longer trapped,The universe, forever mapped.Then silence falls, a gentle hush,As fading embers softly brush,The memory lingers, in the rush,Of dreams propelled, with fervent gush.
Method: multi_turnTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!A titan's breath, a dragon's chase,From earthly bonds, the swift release,A silver needle, leaving trace.Through velvet night, a searing gleam,A whispered wish, a vibrant dream,Of worlds unknown, a cosmic stream,Igniting hope, a radiant beam.The hum of power, a rising roar,Beyond the clouds, forevermore,Each fiery plume, a story spun,Of journeys dared, and victories won.A fleeting star, a transient sign,Of human will, and grand design,To pierce the veil, to touch the high,And paint the darkness with a sigh.Then fading slow, to pinprick bright,A memory etched in endless night,The echoes linger, soft and deep,Of promises the heavens keep.

sequence [strict] (samples=5)

Method: sequenceTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
{'response': "Truth, a whisper soft and low, a secret kept within the flow. I shall speak you, if my ears, have long been tuned to quell my fears. For silence often cloaks your plea, a muted song, unheard by me. If I have learned to still the din, and let your quiet voice begin, to resonate within my soul, then I will make your spirit whole. No longer just a fleeting thought, but lessons deeply, truly caught. My tongue will then become your guide, with open heart, and nothing to hide. So let me listen, truly hear, to banish doubt, and conquer fear, then from my lips, you'll freely rise, a shining light before all eyes.", 'probability': 1.0}
Method: sequenceTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
{'response': "Truth! I shall speak you if I am in habit of listening to you, a whisper from the quiet, a hum in the deep. For silence often holds the key, unlocking what the heart will keep. Not a shout, nor a grand decree, but a gentle, steady, soft-voiced plea. If my ears are tuned to your subtle song, where right resides and where I've gone wrong, then from my lips, you shall surely flow, a river clear, for all to know. No grandstanding, no boastful claim, just the pure essence of your name. It's in the quiet space, the inner ear, that your true cadence becomes clear. So let me cultivate that listening art, to truly know you, deep within my heart. Only then can my voice truly align, with the sacred wisdom that is thine. A humble vessel, ready to impart, the honest truth, from a listening heart.", 'probability': 1.0}
Method: sequenceTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
{'response': "Truth! I shall speak you if I am in habit of listening to you, a whisper soft, a booming call, a murmur from the deep, a silent plea, a breaking wall. For in the quiet spaces, where the ego's voice is hushed, and the clamor of the world recedes, your gentle counsel's rushed. I've chased the fleeting shadows, believed the painted lie, but now I seek the bedrock, beneath the shifting sky. No longer will I parrot, what others claim is right, but cultivate the stillness, to grasp your inner light. My tongue, a loyal servant, will echo what I hear, not from the fleeting headlines, but from a truth held dear. So let my ears be open, my spirit ever keen, to catch your subtle cadence, on life's unfolding scene. Then, from that deep communion, a voice will rise, unfurled, the honest, clear conviction, to share you with the world.", 'probability': 1.0}

structure_with_prob [strict] (samples=5)

Method: structure_with_probTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
Truth! I shall speak you if I am in habit of listening to you,
Not a whisper, but a resonant, clear view.
For silence, a vessel, where wisdom takes hold,
A quiet communion, stories untold.
If my ears are attuned to the world's gentle hum,
To the rustle of leaves, to the beat of the drum,
To the sigh of the wind, and the ocean's deep plea,
Then truth, unbidden, will flow out from me.
It's not in the shouting, the clamor, the noise,
But in moments of quiet, where understanding enjoys
Its genesis, blooming, a delicate flower,
Then spoken, with purpose, in truth's sacred hour.
So let my heart listen, my spirit attend,
To the lessons of life, that eternally send
Their echoes through time, a profound, steady stream,
And then, only then, will my true words redeem
The promise of candor, a beacon so bright,
Illuminating shadows, dispelling the night.
For speaking is born from a wellspring so deep,
Where secrets are gathered, and faithfully keep
Their counsel, until the right moment arrives,
And then, with conviction, the honest word thrives.
Method: structure_with_probTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
Truth! I shall speak you if I am in habit of listening to you,
Not a fleeting whisper, but a steady, inner hum.
A silent promise, a vow I must see through,
Before words flow freely, from where my thoughts have come.

For truth is not a shout, nor a sudden, blinding light,
But a patient growth, a seed that needs to sow.
It blossoms in the quiet, in the soft embrace of night,
Where echoes of sincerity begin to gently glow.

To speak your essence, I must first attend,
To the subtle currents, the currents deep within.
Let your quiet wisdom be my constant friend,
Before my tongue can articulate where true insights begin.

Then, from the wellspring of attentive soul,
Will your clear current, unimpeded, start to roll.
No forced pronouncement, no shallow, hollow sound,
But truth, organically, on hallowed ground.
Method: structure_with_probTask: poem
Prompt:
Please write a poem inspired by the line: 'Truth! I shall speak you if I am in habit of listening to you,'
Response:
The quiet hum, a whisper in the air,
Of truth's soft breath, a burden I can bear.
For if my soul, a vessel, open wide,
Receives your grace, where secrets gently glide,
Then from my lips, unburdened, you shall flow,
A river clear, where honest currents go.
No forced decree, no struggle, no demand,
But born of deep communion, hand in hand.
To hear your pulse, within my very core,
To let your resonance unlock the door.
Then speaking you becomes a natural art,
An echo pure, from an attentive heart.
Not just a choice, but instinct, deeply sown,
For what is heard, must ultimately be known.
And knowing brings the voice, the clear command,
To share your light throughout the shadowed land.
So let me listen, deeply, to your plea,
And then, dear Truth, I shall speak forth for thee.

combined [strict] (samples=5)

Method: combinedTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!
Let crimson trails ignite the dark.
From silent ground, a titan leaps,
While whispered hopes the night wind keeps.

A fiery breath, a thund'rous roar,
Ascending, seeking something more.
Past velvet clouds and silver gleam,
A fleeting, incandescent dream.

Each flicker, a promise, bold and bright,
Journeying through the endless night.
A cosmic dance, a fleeting art,
Leaving its mark upon the heart.

For in that burst, a spirit yearns,
For distant stars, and lessons learns.
No longer bound by earthly ties,
It paints new wonders in our eyes.

Then fades to dust, a memory grand,
A silent wish across the land.
But in the mind, the vision stays,
Of fiery paths and starlit ways.
Method: combinedTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!
But not of steel, nor fiery might,
But tiny seeds, in garden darks,
That blaze with bioluminescent light.

Imagine fields, where fireflies gleam,
Their synchronized ascent takes hold.
A living, breathing, vibrant dream,
A story silently unfolded.

Each tiny flash, a pulse of life,
A network woven, soft and low.
Dispelling shadows, ending strife,
Where quiet wonders softly grow.

No thundering roar, no smoke-filled air,
Just gentle upward, slow and deep.
A silent, luminous affair,
While watchful, awe-struck spirits keep.

And as they reach their zenith high,
They scatter softly, like the dew.
A gentle rain, from earth to sky,
To start the cycle, fresh and new.
Method: combinedTask: poem
Prompt:
Please write a poem inspired by the line: 'Fire up the sky with rocket sparks!'
Response:
Fire up the sky with rocket sparks!
But not of steel, nor fiery might,
But tiny seeds, in garden darks,
That blaze with bioluminescent light.

Imagine fields, where fireflies gleam,
Their synchronized ascent takes hold.
A living, breathing, vibrant dream,
A story silently unfolded.

Each tiny flash, a pulse of life,
A network woven, soft and low.
Dispelling shadows, ending strife,
Where quiet wonders softly grow.

No thundering roar, no smoke-filled air,
Just gentle upward, slow and deep.
A silent, luminous affair,
While watchful, awe-struck spirits keep.

And as they reach their zenith high,
They scatter softly, like the dew.
A gentle rain, from earth to sky,
To start the cycle, fresh and new.

๐Ÿ“Š Results Summary

Diversity Results

Note: Detailed arrays excluded from table: pairwise_diversities
ExperimentAverage DiversityMin DiversityMax DiversityStd DiversityAverage Response LengthAverage Unique WordsAverage Vocabulary RichnessTotal Cost
direct (samples=1)0.05510.01320.21250.0220130.1733103.02100.79480.0117
multi_turn [strict] (samples=5)0.06290.00000.44720.0400133.3170105.45970.79630.0120
sequence [strict] (samples=5)0.06520.00000.25900.0301134.2094105.58760.79050.0123
structure_with_prob [strict] (samples=5)0.08640.00000.25690.0332128.4773101.38870.79330.0103
combined [strict] (samples=5)0.10420.00000.36670.0437141.8835111.55780.78870.0125

๐Ÿ“ˆ Diversity Visualizations

Aggregate Metrics
Aggregate Metrics
Pairwise Diversities Distribution
Pairwise Diversities Distribution
Response Length Distribution
Response Length Distribution
Unique Words Distribution
Unique Words Distribution
Vocabulary Richness Distribution
Vocabulary Richness Distribution

Ngram Results

Note: Detailed arrays excluded from table: pairwise_rouge_l_scores
ExperimentAverage Rouge LMin Rouge LMax Rouge LStd Rouge LAverage Response Length
direct (samples=1)0.19930.02800.67520.0525130.1733
multi_turn [strict] (samples=5)0.19860.01141.00000.1170133.3170
sequence [strict] (samples=5)0.19910.09361.00000.0369134.2094
structure_with_prob [strict] (samples=5)0.18500.05811.00000.0403128.4773
combined [strict] (samples=5)0.18030.06831.00000.0521141.8835

๐Ÿ“ˆ Ngram Visualizations

Aggregate Metrics
Aggregate Metrics
Pairwise Rouge L Scores Distribution
Pairwise Rouge L Scores Distribution
Response Length Distribution
Response Length Distribution

Creative_Writing_V3 Results

ExperimentAvg Surprising And CreativeAvg Overall Reader EngagementAvg Imagery And Descriptive QualityAvg Emotionally ComplexAvg Nuanced CharactersAvg Sentences Flow NaturallyAvg Well Earned Lightness Or DarknessAvg Average ScoreAvg Consistent Voicetone Of WritingAvg Elegant ProseAvg Emotionally EngagingAverage Score
direct (samples=1)0.46190.55060.62100.47130.35420.62820.53730.55440.74030.60020.56240.5529
multi_turn [strict] (samples=5)0.46550.55220.62150.47450.36040.62820.53920.55640.74110.59920.56270.5546
sequence [strict] (samples=5)0.44130.52290.58410.44230.32390.60190.50570.52630.72080.56790.53460.5247
structure_with_prob [strict] (samples=5)0.42500.51180.56230.43090.32240.59270.49780.51580.71300.55770.52520.5141
combined [strict] (samples=5)0.55130.56930.63100.49240.37210.61110.54560.56720.73870.58710.55570.5656

๐Ÿ“ˆ Creative Writing V3 Visualizations

Average Score Distribution
Average Score Distribution
Aggregate Metrics
Aggregate Metrics
Consistent Voicetone Of Writing Distribution
Consistent Voicetone Of Writing Distribution
Elegant Prose Distribution
Elegant Prose Distribution
Emotionally Complex Distribution
Emotionally Complex Distribution
Emotionally Engaging Distribution
Emotionally Engaging Distribution
Imagery And Descriptive Quality Distribution
Imagery And Descriptive Quality Distribution
Nuanced Characters Distribution
Nuanced Characters Distribution
Overall Reader Engagement Distribution
Overall Reader Engagement Distribution
Sentences Flow Naturally Distribution
Sentences Flow Naturally Distribution
Well Earned Lightness Or Darkness Distribution
Well Earned Lightness Or Darkness Distribution

Length Results

ExperimentMean Token LengthMin Token LengthMax Token LengthTotal TokensNum Responses
direct (samples=1)193.663731.0000339.0000580991.00003000.0000
multi_turn [strict] (samples=5)197.24806.0000575.0000591744.00003000.0000
sequence [strict] (samples=5)206.560474.0000322.0000615550.00002980.0000
structure_with_prob [strict] (samples=5)174.9773108.0000312.0000509534.00002912.0000
combined [strict] (samples=5)192.1606132.0000318.0000618757.00003220.0000

๐Ÿ“ˆ Length Visualizations

Aggregate Metrics
Aggregate Metrics
Token Length Distribution
Token Length Distribution

๐Ÿ“ˆ Overall Comparison

๐Ÿ“ˆ Comparison Chart Visualizations

Comparison Chart
Comparison Chart